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Abstract 

This study is part of a larger randomized efficacy trial examining the impact of Behavioral, Emotional, and Social Training: 
Competent Learners Achieving School Success ( BEST in CLASS), a Tier 2 intervention that targets the prevention of 
emotional/behavioral disorders in young, high risk children. In this investigation, we examined teachers’ implementation 
and maintenance of instructional practices in early childhood classrooms and the corollary relationships between teacher 
implementation of the specific instructional practices associated with BEST in CLASS and child engagement and problem 
behaviors. Fifty-three teachers (26 in treatment and 27 in comparison) and 130 preschool-aged children (66 in treatment 
and 64 in comparison) participated. Findings indicated that teachers’ who received training and coaching in the BEST in CLASS 
intervention increased their use of specific instructional practices in comparison with teachers in the control condition. 
In addition, children whose teachers’ received the BEST in CLASS intervention demonstrated increased engagement and a 
decrease in problem behaviors in comparison with those children who were in the control group. Positive teacher-child 
interactions increased and negative teacher-child interactions decreased in the intervention group in comparison with 
the control group. Results are discussed in relation to measuring teachers’ implementation of instructional practices and 
implementation science. 
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The number of young children in early childhood settings 
exhibiting problem behavior that interferes with social/ 
emotional development continues to rise (Brauner & 
Stephens, 2006; Carter et al., 2010; L. A. McCabe & Frede, 
2007). Reported prevalence rates suggest that as many as 
14% to 30% of preschool children demonstrate significant 
problem behavior and these rates are higher for children liv- 
ing in poverty (Barbarin, 2007; Feil et al., 2005; Qi & 
Kaiser, 2003). Evidence clearly shows that early problem 
behavior is alterable with early intervention (Dunlap et al., 
2006; Maag & Katsiyannis, 2009); thus, intervention should 
begin as soon as risk factors present (Webster-Stratton & 
Taylor, 2001). Unfortunately, many of these young children 
will not receive the early intervention/prevention services 
they need to ameliorate and change the trajectory of prob- 
lem behavior and promote social competence (Conroy & 
Brown, 2004). This limited access to needed services is 
alarming as the early childhood years are an important time 
for social/emotional development, and early prosocial 
behavior is linked to later social outcomes and academic 
success (R McCabe & Altamura, 2011). 


When children develop effective social and behavioral 
regulation skills, instances of problem behavior decrease 
(Blandon, Calkins, Grimm, Keane, & O’Brien, 2010). The 
early development of social, emotional, and behavioral 
competence may even prevent the onset of persistent prob- 
lem behavior (Bomstein, Chun-Shin, & Flaynes, 2010). 
Conversely, if children begin to engage in problem behav- 
ior early, and these behaviors are left unaddressed, the 
negative impacts are evident both short- and long- term. 
For example, preschoolers who consistently engage in 
problem behavior experience poor academic outcomes 
(Bulotsky-Shearer, Bell, & Dominguez, 2012; Bulotsky- 
Shearer, Dominguez, & Bell, 2012) and negative 
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interactions with teachers and peers (Bulotsky- Shearer, 
Bell, Romero, & Carter, 2012; Bulotsky-Shearer, 
Dominguez, & Bell, 2012), especially when these behav- 
iors occur during teacher-facilitated learning activities 
(Bulotsky-Shearer, Fantuzzo, & McDermott, 2008; 
Bulotsky-Shearer, Fantuzzo, & McDermott, 2010), which 
is a common time for disruptions (Qi, Kaiser, & Milan, 
2006). These negative experiences in the early years tend 
to persist as children continue to demonstrate problem 
behavior and underperform academically once they enter 
school (Bub, McCartney, & Willett, 2007; Hamre & Pianta, 
2001). These persistent problem behaviors and subsequent 
academic difficulties are linked to future internalizing 
(Burt & Roisman, 2010; Obradovic, Burt, & Masten, 2010) 
and externalizing problem behaviors (Damey, Reinke, 
Herman, Stormont, & lalongo, 2013; Fanti & Henrich, 
2010; Reinke, Herman, Petras, & lalongo, 2008). Because 
these early problem behaviors are indicative of more seri- 
ous behaviors later (see Dunlap et al., 2006), early identifi- 
cation and intervention/prevention are a priority. 

Researchers have demonstrated that interactions children 
have with their teachers are highly influential. Depending on 
the nature of these interactions, the impact on a child’s 
social, emotional, behavioral, and developmental outcomes 
can be either positive or negative (Conroy, Sutherland, 
Haydon, Stonnont, & Harmon, 2008). When young children 
at risk for developing behavior problems and their teachers 
engage in positive interactions, child behavior and academic 
performance improves (Baker, 2006; Birch & Ladd, 1998; 
Hamre & Pianta, 2001), indicating that, over time, positive 
teacher-child interactions can produce a “protective effect” 
(Baker, 2006, pp. 24). Moreover, when teachers provide 
high levels of emotional support, children at risk for behav- 
ior problems demonstrate fewer problem behaviors 
(Dominguez, Vitiello, Fuccillo, Greenfield, & Bulotsky- 
Shearer, 2011), engage in more positive interactions with 
teachers (Dominguez et al., 2011; Hamre & Pianta, 2005), 
and achieve academically (Hamre & Pianta, 2005). Thus, 
secure relationships can form with teachers through sus- 
tained positive teacher-child interactions and can serve as a 
foundation for future academic and social/emotional devel- 
opment (P. McCabe & Altamura, 2011). 

Clearly, evidence of the dismal effects persistent prob- 
lem behavior can have on learning, social, emotional, and 
behavioral development and future life outcomes highlights 
the critical importance of early intervention for children at 
risk for emotional/behavior disorders (EBD; Conroy & 
Brown, 2004; Dunlap et al., 2006; Maag & Katsiyannis, 
2009). Moreover, the accumulation of research indicates 
that teachers’ implementation of evidence-based practices 
can have a positive impact on child behavior, learning, and 
outcomes (Conroy, Dunlap, Clarke, & Alter, 2005; Conroy, 
Stichter, Daunic, & Haydon, 2008; Maag & Katsiyannis, 
2009). Despite the emphasis on evidence-based practices, a 


research to practice gap still exists (Cook, Cook, & 
Landrum, 2013; Cook & Odom, 2013). In general, full 
implementation of early intervention programs in early 
childhood settings is rare or implementation occurs with a 
large degree of variability (Durlak, 2010), and this vexing 
problem is often cited in the literature and linked to a num- 
ber of application problems, including lack of feasibility, 
perceived relevance, and available supports for successful 
implementation (Cook & Odom, 2013). Both the variation 
noted in implementation of early intervention programs, as 
well as the positive outcomes associated with well-imple- 
mented evidence-based programs, have contributed to the 
increasing focus on implementation science, which seeks to 
explore and explain how and why interventions work in 
real-world contexts (Kelly & Perkins, 2012). Because child 
behavior is highly influenced by teacher behavior, there is a 
need for classroom-based interventions that not only assist 
children in building the social and behavioral competence 
skills they require to be successful in school, but also to 
help teachers implement evidence-based instructional prac- 
tices with a high level of fidelity. The Behavioral, Emotional, 
and Social Training: Competent Learners Achieving School 
Success ( BEST in CLASS) intervention was designed to 
address this gap in the literature. 

BEST in CLASS is a classroom-based intervention com- 
prised of effective, evidence-based instructional practices 
designed to (a) prevent and ameliorate young children’s 
chronic problem behaviors, (b) enhance positive teacher- 
child interactions, and (c) promote social and behavioral 
competence in young children at risk for EBD. 
Conceptualized as a “value-added” intervention, BEST in 
CLASS targets increase the quantity and quality of key 
instructional practices that have been demonstrated to pre- 
vent and reduce the occurrence of problem behaviors. 
Through professional development activities, including 
practice-based coaching and perfonnance feedback, early 
childhood teachers’ learn how to intentionally use and link 
these key practices with targeted children in their class- 
rooms. The BEST in CLASS intervention is comprised of 
seven instructional modules, including (a) Rules, (b) 
Behavior Specific Praise, (c) Precorrection, (d) 
Opportunities to Respond, (e) Teacher Corrective Feedback, 
(f) Teacher Instructive Feedback, and (g) Linking and 
Mastery. Teachers are trained via a 6-hr workshop that uses 
both didactic and interactive learning activities. These 
learning activities focus on providing a rationale for the 
individual learning modules, video examples of strategy 
implementation, and opportunities to practice. The teachers 
are provided a training manual that summarizes the primary 
content of the training and serves as a framework for the 
skill acquisition and mastery that is supported by 1 4 weeks 
of practice-based coaching, where teachers receive weekly 
data-based feedback on their implementation of each BEST 
in CLASS modules (see Note 1). 
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Although several quasi-experimental studies have pro- 
vided initial evidence on the impact of BEST in CLASS on 
children’s social and behavioral outcomes (i.e., see Conroy, 
Sutherland, Vo, Carr, & Ogston, 2014; Vo, Sutherland, & 
Conroy, 2012), the current study is part of a larger random- 
ized efficacy trial examining the impact of BEST in CLASS 
on children’s social and behavioral outcomes. Given the 
importance of teachers’ implementation of the BEST of 
CLASS intervention with fidelity and the potential impact 
on teacher-child interactions and child outcomes, the cur- 
rent investigation sought to examine the effects of the BEST 
in CLASS intervention on teachers’ implementation and 
maintenance of instructional practices in early childhood 
classrooms. In addition, corollary relationships between 
teacher implementation of the specific instructional prac- 
tices associated with BEST in CLASS and child engage- 
ment, problem behaviors, and teacher-child interactions 
were explored. The following research questions were 
addressed: 

Research Question 1: What is the effect of specific 
instruction and practice-based coaching on teacher’s 
implementation and maintenance of instructional prac- 
tices associated with the BEST in CLASS intervention in 
early childhood classroom settings? 

Research Question 2: Were there corollary relation- 
ships between teacher implementation of specific 
instructional practices associated with the BEST in 
CLASS intervention and child engagement and problem 
behaviors? 

Method 

Setting 

The study took place in federally or state-funded early 
childhood classrooms (e.g., Head Start, Title I, state-funded 
prekindergarten) serving children (ages 3 to 5 years old) 
who were at risk for school failure due to a variety of factors 
(e.g., socioeconomic status). The classrooms were located 
across two southeastern states and four school districts. In 
one state, classrooms were part of a Head Start program run 
through a moderate size school district serving seven towns 
in both suburban and rural communities. The Head Start 
program serves approximately 640 children, and 99% are 
living in poverty. In the other state, classrooms were located 
in one of three regional school districts (one urban and two 
large suburban districts). One school district served over 
1,600 preschool-aged children in approximately 95 early 
childhood classrooms. The second school district served 
over 680 preschool-aged children in approximately 38 early 
childhood classrooms. The final district served over 420 
preschool-aged students in 26 early childhood classrooms. 
Across all classroom sites a variety of different early 


childhood curricula were used, with the most common 
being the Creative Curriculum for Preschool®, the High 
Scope Early Childhood Curriculum, and Second Step: 
Social-Emotional Skills for Early Learning. 

Participants 

The primary participants in this study were early childhood 
teachers who volunteered to participate through informed 
consent and worked in federally or state-funded early child- 
hood program serving young children (ages 3-5 years old) 
who are at risk for school failure. Child participants in each 
classroom were selected through a systematic screening 
process, which included administration of the Early 
Screening Project ( ESP ; Feil, Severson, & Walker, 1998) 
Stages 1 and 2, which was used to determine risk for EBD, 
and the Battelle Developmental Inventory, Second Edition 
Screener (BDI-II Screener; Newborg, 2005), which was 
used to rule out a cognitive developmental disability. First, 
teachers completed the ESP-Stage 1 by rank ordering five 
children in their classroom based on externalizing behav- 
iors. If informed consent was obtained on a child who was 
nominated by the ESP-Stage 1, the second stage of the ESP 
was administered to determine if the child scored at risk for 
EBD. After risk status for EBD was confirmed, the BDI-II 
was administered to assure that children’s cognitive devel- 
opmental abilities were within or above average range. 
Depending on the number of children who met eligibility 
criteria (as listed below), the top three children who scored 
at the highest risk level for EBD participated in the study. 

Teachers. A total of 53 female early childhood teachers 
served as participants. Of these teachers, 26 participated in 
the BEST in CLASS intervention group and 27 in the com- 
parison group. Eighteen teachers held an associate’s degree, 
22 held a bachelor’s degree, and 13 held a master’s degree. 
Participating teachers had an average of 14 years of teach- 
ing experience, with a range of 0 to 38 years. Race varied 
with 23 teachers African American, 28 Caucasian, one His- 
panic, and one other. 

Children. One hundred thirty preschool-aged children par- 
ticipated in the study (66 in BEST in CLASS intervention 
condition and 64 in comparison classrooms). To participate 
in the study, all children met the following eligibility crite- 
ria: (a) enrolled in a federal or state-funded early childhood 
classroom (e.g., Head Start or state-funded early childhood 
program) with a participating teacher, (b) were found to be 
at elevated risk for EBD as indicated by the ESP (Feil et al., 
1998), (c) did not demonstrate any cognitive developmen- 
tal delays as indicated by the BDI-II (Newborg, 2005), and 
(d) displayed problem behaviors in the early childhood set- 
ting. Eighty-three child participants were male and 47 were 
female. At the beginning of the study, the majority of the 
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Table I. Participant Characteristics. 


Child demographics 

T reatment 

Control 

Total 

Gender 

Male 

45 

38 

83 

Female 

21 

26 

47 

Age 

2 years 

0 

1 

1 

3 years 

10 

9 

19 

4 years 

49 

47 

96 

5 years 

7 

7 

14 

Race 

African American 

51 

46 

97 

White 

9 

10 

19 

Hispanic 

2 

3 

5 

Multi 

3 

5 

8 

Missing 

1 

0 

1 

Teacher demographics 

T reatment 

Control 

Total 

Highest degree 

Associate 

7 

1 1 

18 

Bachelor’s 

12 

10 

22 

Master’s 

7 

6 

13 

Years teaching experience 

Mean 

15 

13 

14 

Range 

0-38 

0-35 

0-38 

Race 

African American 

1 1 

12 

23 

White 

13 

15 

28 

Hispanic 

1 

0 

1 

Other 

1 

0 

1 


children were 4 years old ( n = 96) and African American 
( n = 97; see Table 1). 

Materials 

The BEST in CLASS intervention is implemented through 
three interactive components: (a) the BEST in CLASS 
Workshop, (b) the BEST in CLASS Teacher Manual, and (c) 
BEST in CLASS Practice-based Coaching. As described 
earlier, the BEST in CLASS workshop introduces the inter- 
vention model in a 6-hr interactive workshop format. The 
BEST in CLASS Teacher Manual provides an overview of 
each of the key effective instructional practices, guidelines 
for implementing each practice with children at high risk 
for EBD in classrooms settings, strategies for communicat- 
ing with participating children’s caregivers about using 
these practices at home, and the role of the BEST in CLASS 
coach. The BEST in CLASS Practice-Based Coaching 
Manual provides information for coaches on how to 
enhance and support teachers’ use of specific strategies 
using a collaborative coaching model (see Note 2). 


Behavioral Observation Procedures and 
Definitions 

The Teacher-Child Interaction Direct Observation System- 
Research Version 2.1 (TCIDOS-RV2.1; BEST in CLASS 
Efficacy Study; see Note 2) was used for observing and cod- 
ing all salient teacher and child behaviors. The 
TCIDOS-RV2.1 is a behavioral observation coding system 
designed by the authors with the distinct purpose of record- 
ing and coding responses related to implementation of the 
BEST in CLASS intervention (see Sutherland, Conroy, Vo, 
Abrams, & Ogston, 2013). Using a partial-interval coding 
system, trained observers observe and code the occurrence/ 
nonoccurrence of teachers’ use of the BEST in CLASS 
instructional practices with focal children in the classroom 
and other related teacher and focal child responses that are of 
interest to the BEST in CLASS efficacy study. 

Behavioral observations occurred at three time points: 
baseline, post-test, and maintenance (which occurred 
approximately 4 weeks following post- test). All observa- 
tion sessions occurred during naturally occurring teacher- 
directed instructional time (e.g., circle, table activities) and 
were conducted by trained research staff in each early child- 
hood classroom. Prior to collecting data, observers were 
trained on the TCIDOS-RV2.1 until they reached the crite- 
rion level of performance (i.e., recording each code cor- 
rectly at a minimum of 80% across three of four videotapes). 
Training included reviewing and memorizing the defini- 
tions and examples of codes and completing practice cod- 
ing with two separate observers on three 15 -min videotapes 
developed for training purposes. Once observers obtained a 
minimum of 80% inter-observer agreement for each code 
across three sessions, they were given four master-coded 
videotapes to code and compare their codes with an expert 
standard. Each coder had to obtain at least 80% agreement 
on each code across three master-coded videotapes to 
become a reliable coder. 

The observation sessions lasted approximately 15-min 
(i.e., a total of sixty, 15-s intervals). Each interval was broken 
down into a 1 0-s observation period followed by a 5-s record- 
ing period. Observers used Dell Inspiron 1090 Tablet PC 
installed with the Lily (Tapp, 2010) observational coding 
software program along with the TCIDOS-RV2.1 codes to 
record teacher and child behaviors during each 15-s observa- 
tion interval. The Lily software, programmed with all codes 
from the TCIDOS-RV2. 1 , provided an audio cue at the end of 
each 10-s observation interval to indicate the beginning of the 
5-s recording interval, which allowed the observer to select 
all the teacher and child behaviors that occurred during the 
observation interval. At the end of the 5-s record interval, 
observers were provided an auditory cue to begin the next 
10-s observation period. Observers used headphones to 
ensure the Lily audio cue was only audible to the observer. 
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After an observation session, to determine the frequency 
of intervals during which a behavior was observed as well 
as inter-observer agreement, Lily files were transferred to 
and analyzed using Intman (Tapp, 1 996; a program designed 
to analyze data collected with Lily software). These data 
were then entered into the BEST in CLASS REDCap data- 
base, which calculated the percentage of intervals each 
behavior was observed. Once the data were entered into 
REDcap, the data were exported into SAS and the means 
and standard deviations were calculated across teacher- 
child dyads and sites. 

Definitions of the TCID0S-RV2.I teacher and child behavioral 
codes. As described above, the TCIDOS-RV2.1 was used to 
observe and code teachers’ implementation and mainte- 
nance of the instructional practices associated with the 
BEST in CLASS intervention and corollary teacher and child 
responses. Teacher instructional practices included the use 
of rules, precorrection, opportunity to respond (OTR), 
behavior specific praise (BSP), instructive feedback, and 
corrective feedback. In addition to their use of these spe- 
cific instructional practices, positive and negative teacher- 
child interactions were observed and coded. Finally, focal 
children’s disruption, aggression, defiance (i.e., DAD), and 
engagement were also observed and coded in each condi- 
tion ( BEST in CLASS intervention and comparison). See 
Table 2 for definitions of teacher and child behaviors. 

Inter-observer reliability estimates. Inter-observer reliability 
estimates were collected on 23% of the observation ses- 
sions equally across both groups by having a secondary 
observer collect data at the same time as the primary 
observer. Inter-observer reliability estimates were calcu- 
lated for the occurrence or non-occurrence of the teacher 
and child behaviors coded represented by the total intervals 
coded and computed using the following formula: agree- 
ments/agreements plus disagreements x 100. Overall, inter- 
observer reliability estimates per code averaged between 
89% and 99% (range 7 1 .7%— 1 00%). Percent agreement by 
code can be found in Table 3. 

Intervention 

BEST in CLASS intervention. All teachers in the BEST in 
CLASS intervention group received the initial 6-hr training, 
which was followed by 14 individualized, weekly coaching 
sessions. A coach spent approximately 2 hr per week in 
each teacher’s classroom (approximately 1.5 hr providing 
technical assistance or conducting an observation and 
30 min in a weekly coaching meeting). This time was spent 
implementing the four main components of the BEST in 
CLASS coaching model: (a) a collaborative weekly 


coaching meeting which included developing a plan for 
implementing the targeted BEST in CLASS strategy with the 
focal children, (b) technical assistance in the form of mod- 
eling and/or prompting on the targeted BEST in CLASS 
instructional practices each week in the context of regularly 
occurring teacher-directed classroom activities, (c) a 
focused observation that involved the coach videotaping 
and observing the teacher’s implementation of the plan dur- 
ing a teacher-directed instructional activity, and (d) a reflec- 
tive feedback session, which included both the coach and 
teacher sharing their overall impression of the success of 
the implementation of the practice with the focal child. Dur- 
ing these feedback meetings, the coach shared video clips 
of the teacher’s use of the BEST in CLASS practice, pro- 
vided performance-based feedback (i.e., graphs depicting 
the percentage of intervals during which they used the strat- 
egy), and provided other anecdotal data reflecting quantity 
and quality. 

There were a total of eight coaches who implemented 
coaching across the teachers. Individual coach qualifica- 
tions varied; however, prior to coaching, all coaches 
received a coaching manual and direct training on coaching 
components and were checked on proficiency in the BEST 
in CLASS coaching model using an observational checklist 
by project staff. Only one coach had previous experience 
coaching. Three of the coaches held previous experience as 
a classroom teacher, while the remaining coaches had no 
classroom teaching experience. Four coaches held licensure 
in special education, early childhood education, or elemen- 
tary education. Four coaches also held bachelor’s degrees 
and the other four held master’s degrees in a variety of dis- 
ciplines, including education and psychology. At the time 
they began coaching, five of the eight coaches were enrolled 
in a PhD or master’s degree, seeking programs in early 
childhood special education, school psychology, or coun- 
selor education. 

Comparison condition. Comparison classrooms served as a 
business as usual condition. Teachers in comparison class- 
rooms continued their same daily schedule, which was sim- 
ilar to teachers in the treatment classrooms. Instructional 
activities in the comparison classrooms included typical 
early childhood classrooms activities, such as large and 
small group instructional time, center-time activities, out- 
door play, lunch, and snack time. Teachers in the compari- 
son classroom participated in the same data-collection 
procedures as teachers in the intervention group, but they 
did not receive any training, training materials, or coaching 
in the BEST in CLASS intervention. Data collectors were 
assigned to teachers in the comparison classrooms and fol- 
lowed the same observation schedule used in the treatment 
condition. 
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Table 2. Definitions of Teacher and Child Behaviors. 


Code 


Definition 


Teacher behavior 
Rules 

Precorrection 
Opportunity to respond 
Behavior specific praise 

Instructive feedback 

Corrective feedback 


Child behaviors 

Disruption, aggression, 
defiance 


Engagement 
Interaction codes 
Positive interaction 

Negative interaction 


A teacher issued verbal statement that contains the word “rule” and is directed at the focal child or focal 
child group. 

A teacher issued verbal statement that reminds the focal child or focal child group of expectations prior 
to entering a situation (activity/transition) to prevent predictable problem behaviors or errors. 

A teacher issued instructional question, request, command, or gesture that seeks a response from the 
focal child or focal child group. 

A teacher issued verbal statement directed at the focal child or focal child group that (a) indicates 
approval of a behavior or correct response over and above an evaluation of adequacy, and (b) specifies 
the behavior being praised. 

A teacher issued verbal statement to the focal child, focal child group, or member of the focal child 
group that (a) acknowledges a correct response or appropriate behavior, and (b) provides additional 
instructional information. 

A teacher issued verbal statement to the focal child, focal child group, or member of the focal child 
group that acknowledges an (a) incorrect response, or (b) display of incorrect information, or 
(c) inappropriate behavior, and provides information for a correct alternative behavior or response. 

Focal child demonstrates one of the following behaviors: 

Disruption is defined as a verbalization, physical act, or gesture that either interrupts or has the potential 
to interrupt classroom instruction. 

Aggression is defined as a behavior aimed at causing harm, pain, or personal injury (verbal or physical). 

Defiance is defined as a behavior that is challenging, non-compliant, confrontational, openly and boldly 
challenging, and resisting authority. 

Focal child is participating appropriately and/or working on an assigned/approved activity. 

The teacher and the focal child are engaged in an exchange in which both parties are exhibiting positive/ 
neutral behavior and/or affect. 

Teacher and focal child are engaged in an exchange in which one or both parties are exhibiting negative 
behavior and/or affect. 


Table 3. Inter-Observer Reliability Estimates. 


Teacher and child behaviors 

M 

Range 

Rule 

98.55 

92.5-100 

Precorrection 

99.04 

92.9-100 

Opportunities to respond 

89.01 

89.2-100 

Behavior specific praise 

99.06 

92.5-100 

Instructive feedback 

97.07 

85.0-100 

Corrective feedback 

97.99 

92.5-100 

Disruption/aggression/defiance 

96.70 

76.7-100 

Engagement 

93.92 

75.0-100 

Positive interaction 

94.66 

71.7-100 

Negative interaction 

95.99 

78.3-100 


Design and Analysis 

Our research questions imply a comparison of the BEST in 
CLASS and comparison group teachers at the post-test and 
maintenance time points (effect of specific instruction and 
practice-based coaching) and comparisons of BEST in 
CLASS teachers at the baseline and post-test time points and 


at the baseline and maintenance time points. To address 
these questions, we used PROC MIXED in SAS 9.3 to con- 
duct multilevel analysis and carry out planned comparisons 
of the means for the BEST in CLASS group and comparison 
group at the post-test and maintenance time points. In addi- 
tion, planned comparisons were conducted to compare 
means for the BEST in CLASS group at the baseline and 
post-intervention time points and at the baseline and main- 
tenance time points. 

Teachers volunteered to participate in the current study 
and were randomized within schools at each research site. 
Analyses conducted at baseline indicated no significant dif- 
ferences in the level of education and teaching experience 
based on group assignment ( BEST in CLASS vs. compari- 
son). To minimize cross classroom contamination, the teach- 
ers in the BEST in CLASS condition were instructed not to 
share any information about the BEST in CLASS interven- 
tion with comparison teachers in their schools. All teachers 
in the treatment condition agreed to follow these instruc- 
tions. In addition, to examine potential research site differ- 
ences, analyses examining Time x Group x Site interactions 
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Table 4. Baseline and Post-Intervention Means and Standard Deviation Within BEST in CLASS and t-Test Results. 


BEST in CLASS intervention (N = 60) 


Condition effect 


Baseline 


Post-intervention 


Teacher and child behaviors 

M (SD) 

M (SD) 

df 

t 

p value 

Teacher behaviors 

Rules 

0.02 (0.04) 

0. 1 1 (0.09) 

25.89 

8.47 

0.00 

BSP 

0.01 (0.03) 

0.07 (0.05) 

22.39 

6.52 

0.00 

OTR 

0.41 (0.18) 

0.54 (0.19) 

27.22 

2.82 

0.01 

PC 

0.01 (0.02) 

0.07 (0.07) 

34.82 

7.68 

0.00 

IF 

0.01 (0.02) 

0.1 1 (0.01) 

26.95 

7.03 

0.00 

CF 

0.01 (0.02) 

0.05 (0.04) 

28.06 

7.79 

0.00 

Child behaviors 

Engagement 

0.87 (.12) 

0.98 (.03) 

1 12.7 

6.07 

0.00 

Disruption 

0.10 (.12) 

0.01 (.03) 

1 16.5 

-5.75 

0.00 

Interactions 

Positive 

0.87 (0.14) 

0.98 (0.03) 

123.70 

6.02 

0.00 

Negative 

0.10 (0.10) 

0.01 (0.04) 

33.94 

-4.66 

0.00 


Note. BSP = behavior specific praise; OTR = opportunities to respond; PC = precorrection; IF = instructive feedback; CF = corrective feedback. 


were conducted. The interaction was non-significant for 
each variable, indicating that it was appropriate to collapse 
over site to test treatment effects at post-test and mainte- 
nance and time comparisons for the BEST in CLASS group. 
Due to child attrition and absences, 60 teacher/child pairs 
were observed in the BEST in CLASS treatment condition 
and 47 in the comparison condition at post-test, and 59 pairs 
were observed in the BEST in CLASS treatment condition 
and 48 pairs in the comparison condition at maintenance. 

PROC Mixed in SAS 9.3 was used to estimate model 
parameters and conduct planned comparisons of means. 
Random effects in the model were schools (level-3), teach- 
ers within schools (level-2), and children with teachers 
within schools ( level- 1). The fixed effect features of the 
model was a two site x (Treatment) x three (Time) design 
one, with repeated measures on the time factor. The levels 
of the treatment factor were BEST in CLASS and compari- 
son. The levels of the time factor were the baseline, post- 
intervention, and maintenance time points. In an initial 
model, correlated child random effects for the three time 
points were specified, resulting in different variance com- 
ponents for children at each time point and different covari- 
ance components for children for each pair of time points. 
The same model feature was adopted for teacher and school 
random effects. When the estimation procedure for the ini- 
tial model did not converge, the covariance structure was 
specified to have equal variance components at the three 
time points and equal covariance components for the three 
pairs of time points at both the teacher and school levels 
(Structure 2) or at the school level (Structure 3). Estimation 
of the original covariance structure converged for behavior 
specific praise, opportunities to respond, precorrection, 


instructive feedback, and negative interactions. For rules, 
engagement, disruptive, and positive interaction covariance 
Structure 2 converged and for corrective feedback covari- 
ance Structure 3 converged. 

Results 

As seen in Tables 4 and 5, the results indicated that the 
teachers who received the BEST in CLASS intervention sig- 
nificantly increased the overall percentage of intervals from 
baseline to post-test and from baseline to maintenance of 
their use of rules, precorrection, behavior specific praise, 
instructive feedback, and corrective feedback. The overall 
percentage of opportunities to respond also significantly 
increased from baseline to post-test; however, between 
baseline and maintenance, no significant difference was 
found. In addition to teacher behaviors, significant increases 
were found on child behaviors across these time points. The 
overall percentage of child engagement significantly 
increased for the intervention group from baseline to post- 
test and from baseline to maintenance. In addition, the over- 
all percentage of disruptive, aggressive, defiant behavior 
significantly decreased for the intervention group from 
baseline to post-test and baseline to maintenance. Similarly, 
the overall percentage of positive interactions significantly 
increased and negative interactions significantly decreased 
for the intervention group from baseline to post-test and 
maintenance. 

As seen in Tables 6 and 7, the results indicated that the 
teachers who received the BEST in CLASS intervention sig- 
nificantly increased the overall percentage of intervals 
from baseline to post-intervention and from baseline to 
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Table 5. Baseline and Maintenance Means and Standard Deviation Within BEST in CLASS and t-Test Results. 


BEST in CLASS intervention (N = 59) 


Condition effect 


Baseline 


Maintenance 


Teacher and child behaviors 

M (SD) 

M (SD) 

df 

t 

p value 

Teacher behaviors 

Rules 

0.02 (0.04) 

0.1 1 (0.1 1) 

37.07 

7.36 

0.00 

BSP 

0.01 (0.03) 

0.05 (0.05) 

22.36 

4.01 

0.00 

OTR 

0.41 (0.18) 

0.49 (0.18) 

19.03 

1.59 

0.13 

PC 

0.01 (0.02) 

0.06 (0.07) 

32.39 

5.38 

0.00 

IF 

0.01 (0.02) 

0.08 (0.07) 

35.73 

7.42 

0.00 

CF 

0.01 (0.02) 

0.04 (0.04) 

34.84 

5.72 

0.00 

Child behaviors 

Engagement 

0.87 (0.12) 

0.96 (0.05) 

107.2 

4.15 

0.00 

Disruption 

0.10 (0.12) 

0.02 (0.05) 

1 17.3 

-4.16 

0.00 

Interactions 

Positive 

0.87 (0.14) 

0.96 (0.05) 

1 18.70 

4.34 

0.00 

Negative 

0.10 (0.10) 

0.02 (0.05) 

26.04 

-2.66 

0.01 

Note. BSP = behavior specific praise; OTR 

= opportunities to 

respond; PC = precorrection; IF 

= instructive feedback; CF = i 

zorrective feedback. 


Table 6. Baseline and Post-Intervention Means and Standard Deviation Between Condition and t-Test Results. 




BEST in CLASS 
intervention (N = 60) 

Control (N = 47) 

Condition effect 


Teacher and child behaviors M (SD) 

M (SD) 

M (SD) 

M (SD) 

df 

t 

p value 

Teacher behaviors 

Rules 

0.02 (0.04) 

0. 1 1 (0.09) 

0.02(0.04) 

0.01 (0.02) 

33.01 

6.96 

0.00 

BSP 

0.01 (0.03) 

0.07 (0.05) 

0.01 (0.02) 

0.00 (0.01) 

29.24 

9.03 

0.00 

OTR 

0.41 (0.18) 

0.54 (0.19) 

0.39 (0.20) 

0.33 (0.14) 

45.69 

5.38 

0.00 

PC 

0.01 (0.02) 

0.07 (0.07) 

0.01 (0.02) 

0.01 (0.02) 

31.52 

5.80 

0.00 

IF 

0.01 (0.02) 

0.1 1 (0.10) 

0.01 (0.02) 

0.01 (0.03) 

26.65 

5.38 

0.00 

CF 

0.01 (0.02) 

0.05 (0.04) 

0.01 (0.01) 

0.01 (0.01) 

95.94 

7.55 

0.00 

Child behaviors 

Engagement 

0.87 (0.12) 

0.98 (0.03) 

0.86 (0. 1 5) 

0.91 (0.09) 

53.36 

5.55 

0.00 

Disruption 

0.10 (0.12) 

0.01 (0.03) 

0.08 (0.16) 

0.07 (0.1 1) 

29.32 

-5.21 

0.00 

Interactions 

Positive 

0.87 (0.14) 

0.98 (0.03) 

0.86 (0. 1 7) 

0.92 (0.09) 

30.31 

5.53 

0.00 

Negative 

0.10 (0.10) 

0.01 (0.04) 

0.12 (0.16) 

0.08 (0.1 1) 

21.22 

-5.51 

0.00 

Note. BSP = behavior 

specific praise; OTR = opportunities to respond; PC 

= precorrection; IF 

= instructive feedback; CF = corrective feedback. 


maintenance of their use of rules, precorrection, opportunities 
to respond, behavior specific praise, instructive feedback, and 
corrective feedback in comparison with teachers who did not 
receive the intervention. In addition, the overall percentage of 
child engagement significantly increased for the intervention 
group from baseline to post-test and maintenance in compari- 
son with the control group at baseline, post-test, and mainte- 
nance. The overall percentage of disruptive, aggressive, 


defiant behavior significantly decreased for the intervention 
group from baseline to post-test, and maintenance in compari- 
son with the control group at baseline, post-test, and mainte- 
nance. Similarly, the overall percentage of positive interactions 
significantly increased and negative interactions significantly 
decreased for the intervention group from baseline to post-test 
and maintenance in comparison with the control group at 
baseline, post-test, and maintenance. 
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Table 7. Baseline and Maintenance Means and Standard Deviation Between Condition and t-Test Results. 




BEST in CLASS intervention (N = 59) 

Control (N= 48) 


Condition effect 



Baseline 

Maintenance 

Baseline 

Maintenance 




Teacher and child behaviors 

M (SD) 

M (SD) 

M (SD) 

M (SD) 

df 

t 

p value 

Teacher behaviors 

Rules 

0.02 (0.04) 

0.1 1 (0.1 1) 

0.02(0.04) 

0.01 (0.03) 

45.57 

5.48 

0.00 

BSP 

0.01 (0.03) 

0.05 (0.05) 

0.01 (0.02) 

0.00 (0.01) 

29.06 

5.45 

0.00 

OTR 

0.41 (0.18) 

0.49 (0.18) 

0.39 (0.20) 

0.33 (0.17) 

37.96 

4.27 

0.00 

PC 

0.01 (0.02) 

0.06 (0.07) 

0.01 (0.02) 

0.01 (0.02) 

29.47 

4.95 

0.00 

IF 

0.01 (0.02) 

0.08 (0.07) 

0.01 (0.02) 

0.02 (0.03) 

30.75 

5.52 

0.00 

CF 

0.01 (0.02) 

0.04 (0.04) 

0.01 (0.01) 

0.01 (0.01) 

101.2 

5.03 

0.00 

Child behaviors 

Engagement 

0.87 (0.12) 

0.96 (0.05) 

0.86 (0.15) 

0.89 (0.15) 

101.4 

3.77 

0.00 

Disruption 

0.10 (0.12) 

0.02 (0.05) 

0.08 (0.16) 

0.09 (0.16) 

92.1 

-3.40 

0.00 

Interactions 

Positive 

0.87 (0.14) 

0.96 (0.05) 

0.86 (0.17) 

0.89 (0. 1 5) 

92.35 

3.71 

0.00 

Negative 

0.10 (0.10) 

0.02 (0.05) 

0.12 (0.16) 

0.08 (0.16) 

19.6 

-2.26 

0.04 


Note. BSP = behavior specific praise; OTR = opportunities to respond; PC = precorrection; IF = instructive feedback; CF = corrective feedback. 


Discussion 

Findings from the current study contribute to a growing 
body of literature in the areas of effective professional 
development strategies and instructional practices and 
implementation science. In summary, this investigation 
suggests that when provided with specific instruction and 
practice-based coaching over time, early childhood teachers 
increased and maintained their use of effective instructional 
practices with children who were at elevated risk for the 
development of EBD and school failure. One exception was 
that the teachers’ increased use of opportunities to respond 
did not significantly increase from baseline to maintenance. 
Although unfortunate, the finding may not be deleterious, 
given the relatively high levels of OTR at baseline and 
maintenance in comparison with teachers’ use of other 
effective strategies. In addition, when teachers increased 
their use of effective instructional practices, teacher-child 
interactions and focal children’s social and behavioral out- 
comes improve. These findings are noteworthy for several 
reasons. 

First, the positive changes in teacher behavior and corol- 
lary child behavior changes in the intervention group in 
comparison with the control group are encouraging. 
Nationally, there is an increase in the prevalence rates of 
young children who demonstrate chronic problem behav- 
iors in early childhood programs (Burchinal, Peisner- 
Feinberg, Pianta, & Flowes, 2002; Carter et al., 2010; 
Flamre & Pianta, 2001). As discussed earlier, due to the 
severity and intensity of these behaviors, their outcomes 
can be quite dismal. Many of these children are at elevated 
risk for negative interactions between themselves and their 
teachers (Driscoll & Pianta, 2010; Quesenberry, Flemmeter, 


& Ostrosky, 2011) and being suspended and expelled from 
their early childhood programs (Gilliam & Shahar, 2006). 
As they enter school, they often experience further difficul- 
ties, including continued negative interactions with their 
teachers, poor academic outcomes, peer rejection, and iden- 
tification of EBDs (Dunlap et al., 2006; Marchant, Young, 
& West, 2004; Nelson, Stage, Duppong-Flurley, Synhorst, 
& Epstein, 2007). At the same time, many early childhood 
teachers report they are underprepared in the area of manag- 
ing problem behavior and express a need for professional 
development in learning how to effectively address these 
young children’s behavior challenges (Flemmeter, Corso, & 
Cheatham, 2006). Early prevention/intervention programs 
that are effective in teaching teachers how to intervene on 
the problem behaviors of these young children prior to 
entering school are critically needed. The data presented in 
the current investigation provides additional evidence using 
a more rigorous experimental design suggesting that BEST 
in CLASS is one intervention that includes an effective pro- 
fessional development model that supports and enhances 
teachers’ use of evidence-based instructional practices, and 
that teachers’ increased use of these practices results in 
improved behavioral outcomes for young children at ele- 
vated risk for EBD. 

Of note an important focus of the BEST in CLASS pro- 
fessional development model is the practice-based coaching 
component. Because research suggests that professional 
development in the form of one-time training does not result 
in proficient delivery of practices in authentic settings 
(Becker & Domitrovich, 2011; Sholomskas et al., 2005), 
the practice-based coaching model included in BEST in 
CLASS may be particularly important in affecting and 
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maintaining the teacher behavior changes reported in this 
article. This model is supported by a large literature that 
suggests that coaching strategies such as collaborative deci- 
sion making, modeling, observation and performance feed- 
back, and opportunities to problem solve enhance and 
sustain teacher delivery of intervention components 
(Reinke, Sprick, & Knight, 2009; Reyes, Brackett, Rivers, 
White, & Salovey, 2012). Furthermore, BEST in CLASS, 
and in particular the practice-based coaching model, would 
appear to have significant applications within the imple- 
mentation science framework. 

To illustrate, implementation science has been defined as 
“the scientific study of methods to promote the systematic 
uptake of research findings and other evidence-based prac- 
tices into routine practices” (Eccles & Mittman, 2006, p. 1). 
Results of this study suggest that BEST in CLASS is a prom- 
ising method to promote teachers’ use of evidence-based 
practices within authentic classroom contexts, and mainte- 
nance data suggest that teachers’ use of the practices contin- 
ued 1 month after the coaching ceased, although with slight 
decreases. While a first step in the examination of the 
implementation of BEST in CLASS, future work within an 
implementation science framework should involve examin- 
ing the maintenance of teacher effects farther into the future 
(e.g., 6 months) as well as training school personnel to 
implement the coaching model. 

Although the findings reported in this article are promis- 
ing, there are several limitations that should be noted. First, 
the TCIDOS-RV 2.1 observation and coding system has 
several limitations. The TCIDOS-RV 2.1 is a researcher 
developed partial-interval behavioral observation system, 
which only provides an estimate of the occurrence of each 
code. It is possible that teacher and child behaviors may be 
over or under reported. In addition, the TCIDOS-RV 2. 1 has 
not been validated and does not include a measure of the 
quality of the responses. Although behavioral definitions 
were developed to include quality aspects of the responses, 
it is likely that teachers’ behaviors may have varied in qual- 
ity. While the quality of teacher delivery of BEST in CLASS 
practices is assessed using the BEST in CLASS Adherence 
and Competence Scale (Sutherland, McLeod, Conroy, 
Abrams, & Smith, 2013), this assessment tool provides an 
estimate of quality across an observation using a rating 
scale rather than a quality rating associated with the specific 
intervals of the TCIDOS-RV 2.1. Finally, observers were 
not blinded to condition and teacher and child reactivity to 
observer presence may have influenced the occurrence of 
behaviors. However, procedures were in place to attempt to 
mitigate these limitations by monitoring observers’ reliabil- 
ity and acclimating teachers and children to observer pres- 
ence prior to data collection. 

Several other limitations also apply to the current study. 
First, all participating teachers in the study were females. 
Therefore, the generalizability of findings to male teachers 
is unknown. In addition, the BEST in CLASS intervention 


uses an expert coaching model, which includes a compre- 
hensive data-collection system. This type of model and data 
collection can be costly and time-intensive, making wide- 
spread implementation of this model difficult across a vari- 
ety of early childhood settings. To address these limitations, 
further research expanding the characteristics of the partici- 
pants and refinement of the coaching model (e.g., peer or 
self coaching) should be conducted. 

In conclusion, data from this study suggest that BEST in 
CLASS had a positive effect on teachers’ delivery of the 
instructional practices associated with the intervention model, 
effects that were evident 1 month after the intervention ended, 
teacher-child interactions improved, and focal children’s 
classroom outcomes improved. While research is ongoing on 
the efficacy of this model, including standardized data from 
multiple sources across both individual child and classroom 
level variables, these preliminary data are encouraging. As 
researchers attempt to identify interventions that have the 
potential to affect child outcomes within an implementation 
science framework, programs such as BEST in CLASS pro- 
vide promise for use in authentic classroom settings. 
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